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A LOOK AT MOTION IN THE FREQUENCY DOMAIN 
Andrew B. Watson and Albert J. Ahumada, Jr. 

NASA Ames Research Center, Moffett Field, California 94035 


When a phenomenon seems particularly puzzling, it often helps to view it from a new perspective. 
The puzzle we will examine is the visual perception of motion, and the new perspective w 11 be the fre- 
quency domain. Our precedent here is of course the work of Robson and others [Robson, 1966; Blak- 
emore and Campbell, 1969; Campbell and Robson, 1968] who showed that many aspects of spatial 
vision could be better understood in the frequency domain. 

Our plan is as follows. First we will examine the frequency spectra of moving images and note 
some of their essential properties. Second we will show an example of how this perspective can provide 
simple solutions to long-standing problems in motion perception. Finally we will construct a candidate 
motion detector whose behavior is most easily understood in the frequency domain. 

1. Frequency transforms of moving images 

Consider some image moving at Constant velocity along a straight path. The image contrast distri- 
bution can be written 

c(x,y,/) =c(x,y)8(x -r x /)8(y -ryt) (1) 

where x and y are horizontal and vertical image coordinates and i is time. The spatial contrast distribu- 
tion at time 0 is c(x,y), and r x and r y arc the velocity components in x and y dimensions. To obtain 
the spatiotcmporal frequency spectrum of this image we take the three-dimensional Fourier transform 
of Eq. 1, 

oo oo oo 

C(u,v,w)=f J J c(x,y,t)e~‘ 1 2 "^ xu+)n,+tw ^dx dy dt (2) 

- 00 - 00-00 

where u, v, and w are horizontal spatial, vertical spatial, and temporal frequency respectively. A little 
effort will show that this spectrum is 

C(u,v,w)=C(u,v)S(w+ur x +vry ) (3) 

This function is simple, but somewhat difficult to picture, so we consider for the moment the case in 
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which motion and spatial variation occur only along the x axis, in which ease we can ignore the y 
dimension. The contrast distribution and frequency spectrum for this ease arc shown in Fig. 1. 

Note the following properties of this spectrum: 

(1) it lies along a line of slope -1/r in the u,w space. Thus the higher the spatial frequency com- 
ponent of the image, the higher its temporal frequency. The higher the velocity, the shallower is 
the slope of the spectrum. 

(2) It is symmetric about the origin, and occupies two diagonally opposite quadrants. If motion is to 
the right, these are the even quadrants, if it is to rhe left, they are the odd quadrants. 

We will now show how this spectrum provides insight into motion phenomena. 


2. Stroboscopic apparent motion 

If an image is presented briefly and rapidly at a sequence of closely spaced positions, it may appear 
indistinguishable from a smoothly moving image. This phenomenon of stroboscopic apparent motion 
underlies movies and teltvision, and is a subject of enduring interest in perceptual psychology. We will 
shew that it is easily explained in the frequency domain. 

The strobed image can be viewed as a time-sampled version of a corresponding smooth motion. 
For example, the contrast distr'bution of a thin line in smooth horizontal motion at velocity r can be 
written 


and its spectrum. 


c(x,l)=S(x-rl ) 


C(w,H')=6(»v+m) 


(4) 

(5) 


These two functions are shown in Fig. 2. In keeping with our earlier observations, the spectrum lies 
along a line with slope -1/r in the u,w space. Since the line is thin, we describe it as an impulse func- 
tion. This simplifies the spectrum, making it into a line impulse function. A stroboscopic version of 
this moving line is accomplished by presenting a sample every At seconds with contrast At, 

00 

c s (x,t)=S{x-rt)At 2 8(t-iAt) (6) 

<=-oo 


This function is pictured in Fig. 3. Note that the time interval between samples is At, and the distance 



Fig.l. Contrast distribution (a) and frequency spectrum (b) for a moving image. In this example, a 
Gaussian bar moves to the right. 
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Fig.2. Contrast distribution (a) and frequency spectrum (b) for a thin vertical line moving smoothly to 
the right at velocity r. Both functions are line impulses, whose amplitude dimension should be ima- 
gined to project out of the page. 
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Fig.3. Contrast distribution (a) and frequency spectrum (b) for a thin vertical line in stroboscopic mo- 
tion to the right at velocity r. 

between samples is rAt. The sampling frequency w s is the inverse of the time between samples, 
w s =1/A /. The spectrum of stroboscopic motion is given by 

oo 

C s (u,w)= 2 8(w+ur~iw s ) (7) 

f =—00 

and is shown in Fig. Ob. Note that is is identical to the spectrum for smooth motion except for the 
addition of replicas of the spectrum at intervals of w s . 

At this point we consider the sensitivity of the human eye to stimuli of particular spatial and tem- 
poral frequencies, that is to stimuli that lie at particular points in u,w space. It has been known since 
Shade's work [Shade, 1956] that the human eye is not equally sensitive to all spatial frequencies, and 
that sinusoidal targets above a critical spatial frequency are invisible. Similarly, dc Ixinge [de Lange, 
1954] showed that temporal fluctuations more rapid than a critical temporal frequency are not seen. 
We will call these limits to spatial and temporal frequency sensitivity u t and w / respectively. These two 
limits have been shown to be relatively independent of each other: the spatial limit docs not depend too 
much upon the temporal frequency of the stimulus, and vice versa [Robson, 1966, Kocndcrink and van 
Doom, 1979]. This permits us to roughly characterize the limits of human visual sensitivity to spatial 
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and temporal frequencies by a rectangular window of visibility, as pictured in Fig. 4. Components that lie 
within the window may be more or less visible, but those that lie outside the window are invisible. 
This observation suggests the following hypothesis: two stimuli will appear identical if their spectra, after 
passing through the window of visibility, are identical 

Returning to our stroboscopic stimulus, note that the spectrum of the sampled line differs from 
that of the smooth line only by the addition of the parallel replicas at intervals of the sampling fre- 
quency. Thus the conjecture implies that if these replicas lie outside the window of visibility, then the 
smoothly moving line and the sampled line will be indistinguishable. The replicas may be moved out- 
side the window of visibility by either increasing the sampling frequency (which moves the replicas 
farther from the origin), or reducing the velocity (which makes the replicas more nearly vertical). 
More precisely, we note that for any velocity, the critical sampling frequency will be achieved when the 
first spectral repl ca is just touching the corner of the window of visibility, as shown in Fig.4. This will 
occur at a critical sampling frequency w s , given by 

w c =w t + ru t (8) 

Thus the predicted critical sampling frequency is a linear function of velocity, with intercept given by 
the temporal frequency limit and slope given by the spatial frequency limit. 

2.1. An experiment 

We tested this prediction by means of a two-interval forced-choice experiment. One interval con- 
tained a vertical line which moved smoothly to the right or left, the other interval contained a line 
moving at the same velocity but sampled at a rate w s . The observer attempted to pick the interval con- 
taining the sampled version. 

Fig. 5 shows the results. In each case, the critical sampling frequency increases approximately 
linearly with velocity, as predicted by F.q. 8. For both observers, the intercept is at about 30 Hz which 
is a good estimate for the temporal frequency limit ( w t ) under these conditions. The slope of the 
curve, which according to theory is an estimate of the spatial frequency limit (t//), is 6 c/dcg for one 
observer and 13 c/dcg for the other. These are somewhat low for estimates of the spatial frequency 
limit, but arc not unreasonable given the low contrast and brief duration of the frequency component 
presumably serving to distinguish between smooth and sampled versions. Thus the data in Fig.5 sup- 
port our hypothesis that smooth and sampled motion are visually indistinguishable when the spectral 
components that differ between them lie outside the window of visibility. 



Fig.4. The window of visibility. Stimulus components lying outside the window are invisible. One 
spectral replica is shown lying just outside the window, to illustrate a condition in which smooth and 
sampled images would be just indistinguishable. 
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Fig.5. Critical sampling frequency as a function of velocity for two observers. The stimulus was a 
vertical line 50 minutes long by 0.65 minutes wide which moved horizontally. Observers fixated a point 
at the center of the path of travel. Line contrast was 200%, and background luminance was 50 cd m' 2 . 
A session consisted of 25 trials at each of five sampling frequencies, all at a single velocity. Critical 
sampling frequency was defined as the frequency at which the observer was correct 75% of the time. 
The straight lines arc fitted by eye. 

This theory is not confined to the case of a vertical line moving horizontally with fixed velocity. 
We have considered this case only because it gives rise to particularly simple predictions and because it 
is a case much considered in the literature of apparent motion [Kolcrs, 1972; Morgan, 1979], This gen- 
eral notion can be extended to an arbitrary spatial image undergoing an arbitrary transformation over 
time, and the sampling process can be extended to the two spatial dimensions, as well as time. We 
believe that consideration of the visual filtering of sampled displays provides answers to some long- 
standing puzzles in perceptual psychology, and to some modern problems in advanced visual displays 
[Watson, Ahumada, & Farrell, 1983]. 

3. A linear motion sensor 

As a second illustration of the utility of frequency descriptions of motion phenomena we will con- 
struct a mechanism capable of sensing motion in a particular direction. Our argument in the preceding 
section was that what cannot get through the window of visibility cannot influence perception. Now we 
turn to the issue of how information within the window might be used to sense motion. 

The psychophysical literature on motion perception provides good evidence for the existence of 
mechanisms that are selective for direction of motion. For example a stimulus consisting of the sum of 
two oppositely moving images is little more visible than cither image alone, even though the peak con- 
trast of the sum is twice as great [Levinson & Sckuler, 1975; Watson, Thompson, Murphy & Nachmias, 
1980]. Similarly, prolonged viewing of motion in one direction raises thresholds much more for targets 
moving in that direction than in- the opposite direction [Pantlc & Sckuler, 1969], and patterns moving 
in opposite directions can be discriminated at detection threshold [Watson, Thompson, Murphy & 
Nachmias, 1980J. But there have been few efforts to construct plausible models of motion sensors 
capable of accounting for human performance. What models there arc generally rely on autocorrela- 
tion, requiring a nonlinear multiplicative process. These models have also largely neglected the spatial 
dimension. In contrast, the model we propose is linear, and is explicitly described in both spatial and 
temporal dimensions. We offer it as a candidate model for human motion sensors. 
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3.1. The hyperbolic filter 

Before constructing our sensor we digress in order to introduce the notion of a hyperbolic filler, 


with impulse response 

*«=-*- 

(9) 


wx 

The transfer function of this filter is 

H(u)=—i sgn(u) 

(10) 


where sgn is the sign function. This filter has the interesting property of having constant unit gain, and 
a constant phase lag of it 11. We will use this filter twice in the succeeding development, once in the 
space domain, and once in the temporal domain. It is important to realize that although this function is 
neither causal nor physically realizable, an approximation can be constructed which has the appropriate 
properties, yet is causal and realizable. This is illustrated by the approximate hyperbolic filter shown in 
Fig.6. 

3.2. Constructing the sensor 

We have seen that the spectra of moving images lie along lines through the origin in u,w space. 
To sense this motion one might construct a matched filter passing only energy lying along that line. 
We take a different approach. Wc base our model largely upon the properties of certain well-studied 
visual cells in the cortex of the cat and monkey. These simple cells have a spatial weighting function 
that is reasonably well described by the product of a 2D sinusoid and a radially symmetric Gaussian 
function. Wc assume the diameter of the Gaussian at half height is 1.324 times the the period of the 
sinusoid [Watson, 1983]. This diameter determines the selectivity of the cell for spatial frequency, and 
the number we have chosen gives the cell a spatial frequency bandwidth (at half height) of one octave. 
It has also been observed that simple cells occur in pairs [Pollen & Ronner, 1981] with spatial phases 
w/2 apart. If the spatial impulse response of one cell is j(.x), wc can construct the paired cell by appli- 
cation of the hyperbolic filter, yielding h{x)*s(x). (For simplicity wc consider only the horizontal 
dimension of a vertically oriented weighting function.) If wc arbitrarily assume s(.t) is an even function 



Tima (msec) Temporal frequency <Hz) 


Fig.6. Impulse response (a) and amplitude response (b) of the hyperbolic temporal filter. The exam- 
ple shown is an approximation to a hyperbolic filter that is truncated at its onset and near to the origin. 
Its frequency behaviour is not significantly different from the true hyperbolic filter over the range of 
frequencies of interest (1-60 Hz). 
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(this is not critical), then /i(x)M*) is an odd function. These even and odd impulse responses arc 
shown in Fig.7. 

We assume that the two cells share a common separable temporal impulse response /(/). In the 
absence of better information, we model this by a function fit to human temporal sensitivity, as shown 
in Fig. 8. We now apply a hyperbolic temporal filter to the odd pathway. To insure that the filter is 
causal, we must delay its impulse response by an appropriate amount r. The delay of t in the odd 
pathway introduces an additional phase lag which must be matched by the even path, so we put an 
equivalent delay in there. All of the preceding steps are diagrammed in Fig. 9. 

What have we accomplished by these manipulations? The impulse response of our sensor is 
obtained by convolving the impulse responses of all the cascaded elements, and adding those in paral- 
lel, 



Fig.7. Spatial impulse responses of even (a) and odd (b) pathways. Each is the product of a sinusoid 
and a Gaussian with diameter at half height of 1.324 times the period. The phase in (a) is 0, in (b) it 
is ir/2. 




Fig.8. The impulse response (a), amplitude response (b), and phase response (c) of the initial tem- 
poral filter. The amplitude response in (b) has been fit to sensitivities to temporal modulation of a 
sinusoidal 0.5 c/deg grating as measured by Robson (1966]. 
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Fig.9. Block diagram of the linear motion sensor. 

g(xj)=Af)*[s(x)>8(t--r)+s( X )*h( J c)*h{t)*8(t-r)) (11) 

=f(l)*s(x)'8U-r)*l\+h(x)*hU)] 

The frequency spectrum of our sensor is obtained by Fourier transforming Eq. 11, 

G(u,w ) =F(»v)S(w)e w2irwT ll -sgn(w)sgn(w)l (12) 

The modulus of this spectrum is shown in Fig.10. Note that it is non-zero only in the odd quadrants of 
tv,u space. Recall that these are the quadrants occupied by the spectrum of a leftward moving image, 
so our sensor will respond to motion exclusively in a leftward direction. 

An intuitive explanation for this behavior is as follows. As a sinusoidal grating moves over the 
sensor, the response of the even and odd spatial filters will be temporal sinusoids differing in phase by 
plus or minus n/2, for right and left motion respectively. The hyperbolic temporal filter introduces a 
further shift in the odd pathway of m/ 2, for a resulting phase difference of w or 0. When even and odd 
pathways arc summed, the result is twi( e the amplitude of either path for left motion, and 0 for right 
motion. 



Fig.10. Amplitude response of the motion sensor. 
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Wc draw attention to the following features of our sensor. 

(1) The sensor is selective for direction of motion in 2D space, but is not selective for speed. This 
agrees with psychophysical data that indicate that humans are poor at speed discrimination at 
threshold [Watson & Robson, 1981; Thompson, 1982], Above threshold, speed can be judged by 
noting the temporal frequency of the sensor response, or by examining the pattern of activity 
across many sensors. This is in contrast to other models of speed discrimination which assume 
mechanisms tuned for speed. 

(2) The sensor is selective for spatial frequency and orientation. This is consistent with a wide range 
of psychophysical and physiological results [Movshon, Thompson & Tolhurst, 1978a,b; Schiller, 
Finlay & Volman, 1976; Dc Valois, Albrecht & 'Hiorcll, 1982; Dc Valois, Yund & Helper, 1982; 
Campbell & Robson, 1968; Blakcmore & Campbell, 1969; Watson. 1983). In common with 
recent models [Watson. 1983; Sakitt & Harlow, 1982], we assume the visual system to be popu- 
lated with a many sensors selective for different frequencies, positions, and orientations. To 
include our sensor in such a model is only a minor elaboration, rather than the addition of a 
whole new parallel pathway, since the even and odd spatial filters arc already present. 

(3) The preferred direction of motion of the sensor is orthogonal to the preferred orientation of the 
sensor. This is because the spatial phase shift between even and odd weighting functions is along 
the axis of sinusoidal modulation. For a given sensor orientation, sensors for the two opposite 
directions arc created by either adding or subtracting the odd and even pathways. 

(4) The direction selectivity of the sensor is established by the spatial impulse response s(jr). The 
spatial frequency bandwidth, the orientation bandwidth, and the direction bandwidth of the sensor 
arc all inversely proportional to the diameter of the spatial Gaussian. 

We are not aware of any physiological evidence for or against our hypothetical hyperbolic tem- 
poral filter. It should be noted that it can be approximated by more commonplace mechanisms such as 
a delay or a differentiator. We hope in the future to examine in further detail the extent to which our 
sensor can account for the behaviour of direction-selective visual neurons and for the sensory perfor- 
mance of the human observer. 

4. Summary 

Wc have attempted to show how motion phenomena can be usefully examined in the frequency 
domain. Our first example was a demonstration of how the known spatiotemporal frequency limits of 
vision provide a simple explanation of stroboscopic apparent motion. In our second example, wc 
showed how the constraints on the spectra of moving images lead to a sensor capable of sensing motion 
in a particular direction. Wc recommend this perspective to other investigators of visual motion per 
ccption, not as a theoretical panacea, but as another useful analytic tool. 
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